Timbre and affect dimensions: Evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds

摘要

Considerable effort has been made towardsunderstanding how acoustic and structural featurescontribute to emotional expression in music, butrelatively little attention has been paid to the role oftimbre in this process. Our aim was to investigate therole of timbre in the perception of affect dimensionsin isolated musical sounds, by way of three behavioralexperiments. In Experiment 1, participantsevaluated perceived affects of 110 instrument soundsthat were equal in duration, pitch, and dynamicsusing a three-dimensional affect model (valence,energy arousal, and tension arousal) and preferenceand emotional intensity. In Experiment 2, anemotional dissimilarity task was applied to a subsetof the instrument sounds used in Experiment 1 tobetter reveal the underlying affect structure. InExperiment 3, the perceived affect dimensions aswell as preference and intensity of a new set of 105instrument sounds were rated by participants. Thesesounds were also uniform in pitch, duration, andplayback dynamics but contained systematicmanipulations in the dynamics of sound production,articulation, and ratio of high-frequency to low-frequencyenergy. The affect dimensions for all theexperiments were then explained in terms of thethree kinds of acoustic features extracted: spectral(e.g., ratio of high-frequency to low-frequencyenergy), temporal (e.g., attack slope), and spectrotemporal(e.g., spectral flux). High agreement amongthe participants’ ratings across the experiments suggestedthat even isolated instrument sounds containcues that indicate affective expression, and these arerecognized as such by the listeners. A dominant portion(50-57%) of the two dimensions of affect(valence and energy arousal) could be predicted bylinear combinations of few acoustic features such asratio of high-frequency to low-frequency energy,attack slope, and spectral regularity. Links betweenthese features and those observed in the vocal expressionof affects and other sound phenomena are discussed.